Imaginethisscenario:YourcoolmicroserviceworksfinefromyourlocalmachinebutfailswhendeployedintoyourRedHatOpenShiftcluster.Youcannotseeanythingwrongwiththecodeoranythingwronginyourservices,configurationmaps,secrets,andotherresources.But,youknowsomethingisnotright.Howdoyoulookatthingsfromthesameperspectiveasyourcontainerizedapplication?Howdoyoucomparetheruntimeenvironmentfromyourlocalapplicationwiththeonefromyourcontainer?
Ifyouperformedyourduediligence,youwroteunittests.Therearenohard-codedconfigurationsorhiddenassumptionsabouttheruntimeenvironment.ThecauseshouldberelatedtotheconfigurationyourapplicationreceivesinsideOpenShift.Isittimetorunyourappunderastep-by-stepdebuggeroraddtonsofloggingstatementstoyourcode?
We'llshowhowtwofeaturesoftheOpenShiftcommand-lineclientcanhelp:theocrunandocdebugcommands.
StartingthrowawaycontainersonRedHatOpenShiftMostdevelopersthinkaboutcontainersandOpenShiftonlyforrunninglong-livedapplications.Youcreatedeploymentconfigurations,statefulsets,orcronjobsthatstayaliveforever,creatingandre-creatingpodsasrequired.Yourapplicationisalwayson,oratleastonatfixedintervals.
Theocruncommandrunscontainersthatperformasingletaskandthendie.Itcreatesunmanagedcontainers,thatOpenShiftdoesnotreplacewhentheydie.
IoncehadanapplicationthattalkedtoalegacydatabaseoutsideofmyOpenShiftcluster.Theapplicationwasabletoaccessthedatabasefrommylocalmachine,butnotfromOpenShift.IneededtheabilitytotestaccesstothedatabasefrominsideOpenShift.ThiswayIcouldfindoutwhetherIgotthecorrectenvironmentvariables.Iwouldalsoseewhetherthecontainercouldresolvethedatabase鈥檚hostname.Maybetherewasafirewallblockingaccesstothedatabase?
Thisisaperfectscenariofortheocruncommand.Juststartapodrunningthedatabasecontainerimage.Fromthatpod,youcanusethedatabaseclientandOScommandstotroubleshootconfigurationnetworkconnectivity.Afterafewquicktests,youdon'tneedthepodanymore.
$ocrun-ittest--rm--restartNever\--imageregistry.access.redhat.com/rhscl/mysql-57-rhel7bashThepreviouscommandgivesmeaninteractive(-it)Bashpromptonapodnamedtest.OpenShiftneverrestartsthispod(--restartNever)andremovesitwhenterminated(--rm).
TheMySQLdatabaseimagefromRedHat(rhscl/mysql-57-rhel7)providestheMySQLclientandafewotherusefulcommands,suchasdigandhost.Withthis,IcancheckthatIcanresolvetheserverhostname,connecttothedatabase,andverifymyaccesscredentials.
StartingthrowawaycontainersformanagementclientsIcouldstarttheMySQLclient,oranyothercommandavailablefromthatcontainerimage,directlyfromtheocruncommand.Forexample:
$ocrun-ittest--rm--restartNever\--imageregistry.access.redhat.com/rhscl/mysql-57-rhel7\--mysql-umydbuser-pmydbpassword-hmyserver.domain.example.commydbNotetheuseofadoubledash(--)topreventtheocruncommandfrominterpretingthecommandoptionsintendedfortheMySQLclient.Inthepreviouscommand,thereisno--mysqloption;thereisaspacebetween--andmysql.
Asanotherexample,Icouldstartthesamethrowawaycontainerfromthefirstexample,thenuseanotherterminaltocopyaSQLscriptintothecontainerusingtheoccpcommand.ThenIcanruntheSQLscriptusingthethrowawaycontainershell.
BecausetheMySQLclientcantakeSQLscriptsfromthestandardinput,Icouldjustaddinputredirectiontothesecondexampleandbedone.Ijustpopulatedatestdatabase.WhataboutdoingthisfromashellscriptoranAnsibleplaybook,whileIdonotwritethatfancyoperatorthatwoulddeployandinitializethedatabaseformyapplication?
Thankstotheocruncommand,Icanuseadministrationclientsembeddedintomanycontainerimages,forexample,theCLIadministrationtoolsforJBossEAP,AMQ,andsingle-signon.Idonotneedtoinstallanyofthemonmylocalmachine.Cool,isn鈥檛it?
CloningadeploymenttoadebugcontainerIcouldaddmorecommand-lineoptionstotheocruncommandandreplicateallthesettingsofanexistingdeployment:environmentvariables,resourcelimits,andsoon.Ifmyintentweretoreplicatetheruntimeenvironmentofmyapplication,thiswouldbetoomuchworkandbepronetoerrors.
However,thiswouldbeascenariofortheocdebugcommand.Itcreatesanewpodthatisacarboncopyofanexistingpod.Ifyourpoddoesnotstartforwhateverreason,youcancreatethecopyfromitsdeploymentconfiguration.
SupposethatIcreatedmyapplicationusingocnew-appandnameditmyapp.Tocreateadebugpodfromitsdeploymentconfiguration,Iwouldusethefollowingcommand:
$ocdebug-tdc/myappIgetaBashshellrunningunderthesamerestraintsasmyapplication:uid,SElinuxcontext,environmentvariables,andthesamecontainerimage.
IfIsuspectthatsomeoftheserestraintsmaybecausingafailure,Icanselectivelyoverridethemusingoptionsfromtheocdebugcommand.Forexample,addingthe--as-rootoptiontothepreviousexamplegivesmearootpromptinsidethepod,butonlyifmyOpenShiftuserhasaccesstoasecuritycontextconstraintthatallowsmetodoso.
Thedebugpodrunswithhealthprobesdisabled.Icanstartmyapplicationmanuallytocheckwhetherthehealthprobesareincorrectandforcingmypodtoterminate.Icouldaddoptionstotheocdebugcommandtoenablehealthprobes,disableinitcontainers,disablesidecarcontainers,changelabelsthataffectpodscheduling,andthusfindwhich,ifany,ofthedeploymentsettingsarenotcorrectformyapplication.
StartingthrowawaycontainerswithRHELtoolscontainerimagesAswiththeocruncommand,youractionsusingtheocdebugcommandarelimitedbywhatisincludedwithyourapplicationcontainerimage.Fortunately,youcanoverridethecontainerimageinyourdebugcontainer.Goodcandidatesaretherhel7/toolsandtherhel8/support-toolscontainerimagesfromRedHat.
$ocdebug-tdc/myapp\--imageregistry.access.redhat.com/rhel7/rhel-toolsTheseimagesprovideaccesstostandardRHELtroubleshootingcommandsthatwouldnotbeincludedinmostapplicationimages,forexample,thepinganddigcommands.
You'llneedtodownloadtherhel8/support-toolscontainerimagefromtheRedHatterms-basedregistry(redhat.registry.io).Accesstotheterms-basedregistryrequiresapullsecret.FollowtheinstructionsfromRedHatEnterpriseLinuxSupportToolsifneeded.
ConclusionYoudonotneedalocalcontainerenginetorunthrowawaycontainersthatperformtroubleshootingandone-timetasks.YoucanrunthesecontainersquicklyandeasilyonRedHatOpenShiftusingtheocrunandocdebugcommands.And,yourOpenShiftcluster,ifitisnotaMinishiftinstance,isprobablyquickertodownloadcontainerimagesandlikelyhasmorestoragespaceandbetterbandwidththanyourlocalworkstation.
Lastupdated:September3,2019